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Title: Improvements in and relating to imagine 
Field of invention 

This invention concerns a method and apparatus by which luminescent elements of a 
specimen or sample can be imaged for analysis. 

Background 

It is known to use laser excitation confocai imaging, a Nipkow disc scanner (which may be 
a microlens enhanced disc scanner) and a digital camera, to provide image data in two 
dimensions, and if controlled Z-axis movement is also provided, three dimensional image 
data can be obtained. Where a luminescent element is altering with time, a sequence of 
images can be obtained to provide data in four dimensions, (ie area, depth and time). 
Where possible, other parameters may also be changed or observed, including excitation 
wavelength, emission wavelength, polarisation, and/or emission lifetime. 

One system which is capable of providing such data about microscopic specimens such as 
cells and tissue samples is the UltraVIEW live Cell Imager as produced and supplied by 
Perkin Elmer Life Sciences. This system uses a confocai laser microscope and CCD 
camera imaging system such as is described in a paper written by Kawamura Negishi 
Otsuki and Tomosada entitled Confocai Laser Microscope and CCD Camera, published in 
the Yokogawa Technical Report No. 33 (2002), English Edition and Yokogawa Patent No. 
EP 0539691 WO 9804946. 



It is also known to use one or more lensless discs to achieve confocality as described in US 
6147798 to Atto. Here rotation of the lensless disc or discs correspond to the rotation of 
the Nipkow disc(s). 
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US S248876 and WO 03/019242 describe another confocal scanning process in which a 
fixed pattern of pinholes are employed to scan in a linear manner. One complete scan 
using the pinhole array corresponds to rotation of the Nipkow disc(s). 

US S034613 (Denk) describes another process^ known as a 2 photon process, to limit the 
focal plane of activation and achieve confocality. Here one pulse from the excitation 
source equates to rotation of the Nipkow disc(s). 

Another pinhole-based confocal scanning system is described in 1998 Opt-Leth 23(9): 655- 
657 in a report entitled Multifocal Multiphoton Microscopy by Beresdorf et al. Here one 
complete scan using the pinholes corresponds to rotation of the Nipkow disc(s). 

A further confocal scannmg system is described in US 2002/024007, EP 0911667, US 
5,587,832 and US 6,483,641/EP 1207415. This system employs a controllable array of 
mirrors to synthesise an array of pinholes. Again one complete scan of the pinholes 
corresponds to rotation of the Nipkow disc(s). 

Lastly transmission and reflective elements arranged in a random or quasi-random pattern 
can be used for confocal scanning as described in WO 97/31282 and WO 0043819 (Wilson 
et al). Again one complete scan of the transmission and reflective elements corresponds to 
the rotation of the Nipkow disc(s). 

It is an object of the present invention to provide an imaging system which is an 
improvement over the systems currently in use and in particular over the current 
Ultra VIEW system, and that described in the aforementioned Yokogawa Technical Report 
No. 33 (2002) and related Patents EP 0539691 and WO 9804946, EP 1245986. US 
5,633.751 and US 6,388.808. 

It is also an object of the present invention to provide a control system for more accurately 
controlling the imaging and data capture process and a method of operation of such a 
system. 
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It is also an object of the present invention to provide an improved method of capturing 
light from luminescing elements in a sample or specimen and converting the captured light 
into data for processing for analysis and/or display and/or storage for subsequent display 
and/or processing for analysis. 

Siimmarv of the invention 

According to one aspect of the present invention in a method of imaging light from a 
specimen in which excitation light passes to the specunen via a scanning system and light 
emitted by luminescence of the specimen passes in another direction via the scanning 
system to an image capture device having discrete spatially distinct light sensitive regions 
wherein the scanning system is operated so as to scan the whole of an area of interest of 
the specimen the excitation light and/or the image capture device are controlled so that 
light emitted from the specunen is only incident on the image capture device for a specific 
time period equal to that required for scanning the whole of the area of interest n times 
(where n is a whole number equal to or greater than 1). 

The area of interest may be the whole of the viewable area of the specimen or a part 
thereof, and a window of variable size and location may be created by the scanning system 
to enable some or all of the area to be scanned, and for example a subarray of variable size 
and location may be used. 

In a preferred arrangement the scanning system comprises a rotating a Nipkow disc 
scanner in which excitation light passes through openings in the disc in one direction and 
light emitted by luminescence of the specimen passes through the openings in the opposite 
direction to form an image at the image capture device, and the pattern of openings is such 
that rotation of the disc through results in scanning the whole of the area of interest and 
ttie specific time period is selected to correspond to nA° of rotation of the disc (where n is 
a whole number equal to or greater than one). 
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Conveniently the image capture device is a CCD camera. 

A single Nipkow disc may be employed, but more preferably a 2-disc arrangement is 
employed, one containing openings and the other microlenses, as described in the 
Yokogawa Technical Report No. 33 (2002). References herem to a disc (or disc means or 
disc arrang«nent) are intended to include multiple disc assemblies such as described in that 
Report, as well as single disc arrangements, where the context so admits. 

Alternatively one or more lensless discs may be used (for example as described in US 
Patent 6147798). In this case, the rotation of the lensless disc corresponds to rotation of 
the Nipkow disc. 

Alternatively, light may be scanned using a linear process such as a raster scanning 
technique, using one pinhole, or several pinholes in a fixed pattern (for example as 
described in US Patent 5248876 also WO 03/019242) where one complete linear scan of 
the pinholes corresponds to rotation of the Nipkow disc. 

Alternatively a scanning slit scanner may be used (for example as described in US 
6038076) where a scan of the slit corresponds to rotation of the Nipkow disc. 

Alternatively, confocality may be achieved by a 2 photon process which limits the focal 
plane of activation (see as described in US Patent 5034613) where one pulse of the 2 
photon excitation source corresponds to rotation of the Nipkow disc. 

Alternatively confocality may be obtained by a time delay multiplexed process (as 
described by Bewersdorf, Pick and Hell in a paper entiUed "Multifocal Multiphoton 
Microscope" 1998 Opt Lett. 23(9): 655-657) where one complete scan of the pinholes 
corresponds to rotation of the Nipkow disc. 
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Alternatively an array of independently controllable mirrors may be used to synthesise the 
pinholes (see for example EP 0911667 and US 2002,024007) where a complete scan using 
the mirrors, creates an image or subimage and corresponds to rotation of tlie Nipkow disc. 

Alternatively a random or quasi random pattern on transmissive and reflective elements 
may be scanned (as described in WO 97/31282 and WO 0043819) where one complete 
scan of the transmissive and reflective elements corresponds to rotation of the Nipkow 
disc. 

According to another aspect of the present invention, in a method of imaging light from a 
specimen using an image capture device comprising a sensor containing discrete spatially 
distinct light sensitive regions, a scanning device is employed by which light is prevented 
from reaching the specimen except from an excitation source for each of a succession of 
separated specific periods of time, the image capture device is read out at the end of each 
said period of time and is reset prior to the beginning of the next said time period, and 
light from the source of excitation light is supplied for a tune precisely equal to each said 
specific period of time and thereafter is extinguished, so that the duration of the image 
capture device exposure is controlled by the time for which the excitation light source is 
active. 

In one method embodying the invention the excitation light source is switched on and off to 
control the duration of the exposure. 

Alternatively or additionally, shutter means may be provided to interrupt light from the 
excitation source except for when the specimen is to be scanned, and the excitation light 
source may be operated continuously or for longer periods of time than the said specific 
periods of time. 

Alternatively or additionally an intrinsic (electronic) shutter may be used. 

Alternatively or additionally a light chopper may be used to interrupt the excitation light. 
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Alternatively an acousto-optic light intemipting element may be used. 

Shutter means may be provided between the scanning system and the image capture device 
to prevent light reaching all or part of the capture device sensor, except for the specific 
periods of tune during which the excitation light is incident on the specimen, thereby to 
reduce errors which could arise from light arising from phosphorescence, afterglow, stray 
reflections or other effects, from reaching the device. 

If shutter means is provided for both excitation source and image capture device, they are 
preferably synchronously operated such as by mechanical or electrical means, or electronic 
pseudo shutter means may be employed or any combination thereof. 

Where the specimen is largely transparent it can be scanned and imaged at different spaced 
apart planes which are also preferably parallel to one another. 

Multiple images may be collected over short periods of time (eg 1 niinute-60 minutes) or 
extended periods of time (eg 24-72 hours) as well as intermediate periods (eg 1-24 hours). 
Such images are often referred to as a stack of images. 

The method may thus involve relative movement between the specimra witii and the 
scanning device. This may be achieved for example by displacing the scanning device 
along an axis (typically its Z axis) which is normal to the scanning plane to be imaged, so 
that the incident light can be brought to a focus at different points along that axis, each 
point lying in (and defining) a different focal plane, to enable the specimen to be scanned 
and imaged in different parallel spaced apart planes. Preferably movement is restricted to 
periods during which at least excitation light is not incident on the specimen and/or to 
periods during which the image capture device is also insensitive to light from the 
specimen luminescence. 

Alternatively the optical path may be modified to define the different planes, by moving an 
objective lens or the specimen, typically by moving a holder containing the specunen. 
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Thus in a preferred arrangement a specimen carrier is displaced to define the spaced apart 
imaging planes. 

The axis of displacement may be at an angle of other than 90° to the focal plane. 

Typically the Z-axis movement is performed at the end of each said time period, or after N 
successive such time periods (N being an integer equal to or greater than 1). The 
movement may be such as to increase or decrease the distance between the scanning device 
or optical system element and the specimen. 

Each Z-axis movement between exposures may be the same size or may differ from those 
before and/or following. 

Z-axis movement is preferably made in discrete steps, each of a predetermmed equal size 
and in that event, each Z-axis movement between exposures may be made up of M discrete 
steps, (where M is a whole number equal to or greater than 1). 

Alternatively the Z-axis motion may be continuous. This aUows faster motion between Z 
positions without having to wait for the Z mechanism to settfe. The disadvantage is that 
the Z position is varying during the imaging with the result that the image can be blurred 
in Z. However tiiis may be compensated for by applying deconvolution to re-sharpen the 
image. 

Where the time required to alter the Z-axis spacing is greater than die mmimum time 
needed between exposures, the method preferably includes the step of adjusting the time 
between exposures or controlling the Ulumination and image capture device exposure by 
reference to the Z-axis movement, such that excitation Ulumination and exposure is only 
initiated after a sufficient period of time has elapsed or after a desired Z-axis movement 
has been achieved. 

The excitation light may be composed of light having two or more differing wavelengths. 
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Where the method involves variation of the wavelength of the excitation light from one 
exposure to another, the method preferably also includes the step of altering the excitation 
light wavelength between the end of one exposure and the beginning of the next. 

Where the variation of excitation light wavelength is to be effected whilst focussing the 
light onto the same plane of the specimen (ie there is no Z-axis movement between each of 
a succession of two or more excitation and exposure steps making up a sequence of 
exposures, albeit each employing one or simultaneously two or more different 
wavelengdi(s) of excitation light), the Z-axis movement is initiated only after each 
complete sequence of exposures has been performed, and the next sequence of such steps 
is only initiated after each Z-axis movement has been completed. 

Additionally, the stage of a microscope used in the system, and carrying a specimen, may 
effect larger scale X or Y motions (10 micron to lOnun) to bring into view other portions 
of a sample such as for example other cells on the same slide or other regions of tissue or 
sample. 

A number of samples may be held in the wells of a multiwell plate. The samples may all 
be similar but may be different and may for example differ due to constituents, 
concentration or age. The multiwell place may be mounted on a microscope stage which is 
moved so as to bring different wells into registry, or different parts of a single sample in 
one of the wells into registry. 

Additionally, the Z motion may be in one direction and then in another, alternately, to save 
time. 

Wavelength variation may be achieved by employing two or more light sources of different 
peak wavelengths and selectively operating the sources, or selectively directing light from 
the sources in turn, as required, to the scanning device and specimen. 
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The light sources used may have more complex spectral characteristics (eg multiple peaks 
available from an Argon ion laser with a main line at 488nm, and additional lines at 476nm 
and 496nm). 

Where light of more than one wavelength is produced at the same time, the wavelengdis 
may be selected by a single code (eg 2 lines to select one of 4 wavelengths as code 00, 01 » 
10, 11) or as a set of individual signal lines (eg 4 lines for 4 wavelengths). In tliis way 
several lines may be activated at the same time. 

According to a preferred feature of the invention the excitation light may be obtamed from 
one or more laser light sources. 

Alternatively, the light source may comprise a directly driven laser diode. 

In a preferred arrangement a single light source is employed which comprises an acousto 
optic tuneable filter (AOTF) crystal and the wavelength of the emitted light is altered by 
altering the frequency controlling signal to the crystal as required. Such an arrangement 
may contain a single laser source, or two or more laser light sources whose outputs can be 
optically combined. 

Preferably any alteration of the frequency of the controlling signal is effected between 
exposures. 

The excitation light may be pulsed. 
The excitation light may be polarised. 

A luminescent or incandescent light source may be used as the source of excitation light, 
and an optical filter or a monochomator employed to control the wavelength of the light 
emitted by the source. 
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Alternatively a light emitting diode (LED) source may be used which may comprise one or 
more light-emitting elements. 

Where the intensity of the excitation light incident on the specimen is to be adjusted from 
one exposure to another, the method preferably mvolves the step of adjusting the intensity 
of the incident illumination between exposures. 

This may for example be achieved by interposing neutral density filters, openmg or closing 
an iris diaphragm in die light path, adjusting the power to the light source, employing an 
attenuating element such as an AOTF or LCD shutter or any combination thereof. 

Where the intensity of the excitation light incident on the specimen at one wavelength, is 
different from that at another wavelength, such as may be due to inherent intensity 
variation as between one source and another, or between different modes of operation of a 
controlled excitation light source, the intensity of the excitation illumination between one 
wavelength and another may be adjusted so as to provide a predetermined mtensity of 
illumination at the specimen for each wavelength. 

Typically but not exclusively, the power adjustment from one exposure to another as 
wavelength varies, is such as to ensure a substantially similar level (ie intensity) of 
illumination at the specimen during the scanning making up each exposure, independent of 
wavelength. 

A similar technique may be employed if the emission from the sample differs according to 
wavelength of the luminescence, and in order to render the emitted light of similar 
intensity (irrespective of wavelength), the power adjustment and/or attenuation from one 
exposure to another may also, or instead, be adjusted to provide substantially constant 
intensity luminescence irrespective of the wavelength thereof. 

Preferably a system performing a method embodying the invention is controlled from a 
single control means such as a control centre which typically includes a programmable 
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computer and one or more interfaces for converting signals produced by the computer into 
signals suitable for driving, or controlling, the supply of power to drives, or to control 
power to devices to effect the movements and rotations required by the different steps of 
the method. 

A preferred mage capture device is a CCD camera. Such a camera can provide a picture 
signal m analogue or digital format. Preferably a camera is selected which produces a 
picture signal in digital format. 

Where the image capture device is a CCD camera, it is preferably cooled to increase the 
S/N ratio of the camera output signals. 

Alternative image capture devices which may be employed are a light sensitive sensor, a 
CMOS camera, a CID (charge injection device) camera, an intensified or gated camera, a 
photomultiplier tube array, a photodiode array or an image capture device employing an 
addressable microcolometer array, or a chemical film camera may be used. 

Since the light presented to the image capture device is capable of forming an image, the 
light may be directed instead or as well to an eyepiece which will allow the image to be 
seen by die human eye, or to optical apparatus adapted to project the image onto a screen 
for viewing. 

The light path to ihe image capture device may include a beam-splitting or beam directing 
device by which light can be split (or divided) between the image capture device, an 
eyepiece, or projection system. 

A plurality of image capture devices may be employed each of which is supplied with light 
from the specimen and each of which may be the same type of device or different from the 
other device(s). 

Cameras may be replaced by sensors. 
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Where one or more image capture devices are provided, the light path thereto may include 
a beam splitting means or beam directing means as appropriate. 

The light image may be transmitted to an image capmre device (which may be a sensor) 
via a fibre bundle. 

The signals from the image capture device may be processed and/or analysed in real time, 
for example using a suitably programmed computer. 

Signals from the image capture device may be stored for example in a frame store for 
subsequent analysis by a computer, or may be displayed in real time (or subsequently by 
reading out signals from the store) for visual inspection and analysis of an image produced 
from fhe signals. 

The system may operate using dark field iUumination, or Rheinberg illumination, phase 
contrast illumination, or Differential Interference Contrast (DIC) illumination. 

The system may utilise Hoffman Modulation Contirast may be employed. 

The illumination may be oblique or axial, and tiie illuminating light may be polarised 
and/or from a non-visible part of the spectrum such as infrared light. 

Apparatus for imaging light emitted from a specimen, typically due to fluorescence of 
different parts of tiie specimen, onto an image capture device to produce a video signal for 
creating an image in a display device or for processing and analysis, comprises 

- means for mounting die specimen. 

- a light source for producing excitation light. 
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- a scanning system adapted to scan the specimen with the excitation light (as by directing 
excitation light in one direction towards, and thereby to scan an area of the specimen) and 
also adapted to convey light emitted from the specimen as a consequence of the 
excitation light impinging thereon in another (typically the opposite) direction, which 
operates in use to scan (typically repeatedly) an area of interest of tiie specimen, 

- an image capture device having discrete spatially distinct light sensitive regions on which 
the light emitted from the specimen is focussed to form an image after being conveyed 
(i.e. passing) through the scanning system in the said otiier direction, and 

- control means adapted to control die excitation light and/or tiie image capture device so 
&at light from the specimen is incident on the image capture device for a time period 
equal to that required by the scanning system to scan the area of interest n times (where n 
is a whole number equal i» or greater than 1). 

The area of interest may be the whole of the viewable area of die specimen. Alternatively a 
window of variable size and location may be generated by die scanning system to enable 
part (or if desired all) of the area to be scanned. Thus a sub-array of variable size and 
location may be used. 

The scanmng system is preferably a confocal system 

In a preferred arrangement die scanning system comprises a rotating Nipkow disc scanner 
in which excitation light passes tiirough openings in die disc in one direction and light 
emitted by tiuorescence of die specimen passes flirough die openings in the opposite 
direction to form an image on the image capture device, and die pattern of openings is 
such tiiat rotation of die disc Ourough A« results in scanning die whole of die area of 
interest and die said time period is selected to correspond to nAo of rotation of die disc 
(where n is equal to or a whole number greater tiaan one). 
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A single Nipkow disc may be employed, but more preferably a 2-disc arrangement is 
employed in which one of the discs contains one containing microlenses in the openings, as 
described in the Yokogawa Technical Report No. 33 (2002). References herein to a disc 
(or disc means or disc arrangement) are intended to include multiple disc assemblies such 
as described in that Report, as weU as single disc arrangements, where the content so 
admits. 

Alternatively one or more lensless discs may be used (for example as described in US 
Patent 6147798). In this case, the rotation of the lensless disc (or discs) corresponds' to 
rotation of a Nipkow disc. 

The scanning system may incorporate a pinhole, or several pinholes in a fixed pattern (for 
example as described in US Patent 5248876 also WO 03/019242) where one complete scan 
of the pmhole (or pinhole array) corresponds to rotation of a Nipkow disc. 

Alternatively the scanning system may incorporate a scannmg slit scanner (for example as 
described in US 6038076) where a scan of the slit corresponds to rotation of a Nipkow 
disc. 

Confocality may be achieved by a 2 photon process which includes a 2 photon excitation 
source and which limits the focal plane of activation (see as described in US Patent 
5034613) an wherein one pulse of Ae 2 photon excitation source corresponds to rotation of 
a Nipkow disc. 

Alternatively confocality may be obtained by a time delay multiplexed process involving 
pinholes (as described by Bewersdorf, Pick and Hell in a paper entiUed "Multifocal 
Multiphoton Microscope" 1998 Opt Lett. 23(9): 655-657) wherein one complete scan of 
tiie pinholes corresponds to rotation of a Nipkow disc. 

Alternatively die scanning system may employ an array of independenfly controllable 
mirrors and means for adjusting same under the control of the control means, so that in use 
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they synthesise an array of pinholes (see for example EP 0911667 and US 2002,024007) 
wherein one complete scan using the mirrors to create an image or subimage, corresponds 
to rotation of a Nipkow disc. 

Alternatively the scanning system may include a random or quasi random pattern of 
transmissive and reflective elements (as described in WO 97/31282 and WO 0043819) 
whidi are scanned to produce an image where one complete scan of the transmissive and 
reflective elements correspoiMls to rotation of a Nipkow disc. 

Preferably means is provided to prevent light from reaching the specimen except from the 
excitation source, and to prevent light from reachmg the image capture device except from 
the specimen via the scanning system. 

Preferably the control means also controls the operation of the image capture device 
whereby it is read out at the end of each said time period and is reset prior to die beginning 
of the next said time period. 

Preferably means is provided by which light from die source of excitation light is incident 
on die specimen for a time precisely equal to each specific time period. 

In one arrangement the control means switches die excitation light source on and off to 
control the exposure duration, and the image capture device exposure is in turn governed 
or controlled by die time for which die excitation ligjit source is active. 

Alternatively or additionally, shutter means may be provided which in use is operated by 
signals from tiie control means to interrupt light from die excitation source except for when 
the specimen is to be illuminated. 

Alternatively or additionally die light may be controlled by an intrinsic (electronic) shutter. 
Alternatively or additionally a light chopper may be used to control the light. 
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Alternatively an acousto-optic element may be used to perform the shutter function. 

Alternatively or additionally, second shutter means may be provided between the scanning 
system and the image capture device, which second shutter means is operated by signals 
from the control means so that in use light is prevented from reaching at least part of the 
image capture device sensor, except for the specific periods of time during which 
excitation light is incident on the specimen, for the purpose of reducing errors which could 
arise from phosphorescence, afterglow, stray reflections or light from other effects, 
reaching the capture device. 

If shutter means is provided for both excitation source and image capture device, means is 
preferably provided whereby in use. they are operated synchronously. 

The synchronising means may be mechanical or electrical or both. 

In use the scanning system preferably focuses the excitation light in a plane which contains 
or comprises the area of the specimen to be illuminated by the excitation light, and in 
general comprises the area of interest. 

Where the specimen is largely transparent, the apparatus may also include drive means 
adapted to move the specimen, the scanning system, or an element of an optical system 
within the scanning system, along a linear axis (the Z-xis) so that in use the plane can be 
positioned at different points along the Z-axis, to enable the specimen to be scanned in 
different (typically parallel) spaced apart planes. 

Preferably the linear axis drive means is also controlled by signals from the control means. 

Preferably flie control system operates so that in use linear axis movement is restticted to 
periods during which at least excitation light is inhibited (or prevented from reaching the 
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specimen) and is preferably restricted to periods during which the image capture device is 
rendered insensitive to light. 

Typically the control means controls the linear axis drive in use so as to effect the linear 
axis movement at the end of each said time period, or after a succession of N such time 
periods, (N being an integer equal to or greater than 1). The movements may be such as to 
mctease or decrease the distance between the specunen and the scanning system, or an 
element of the scanning system, or an element of an optical system within the scanning 
system. 

The control system is preferably adapted to control the distance moved along the Imear 
axis between exposures. 

Each movement m the linear axis may be same size or may differ m size from those 
before and/or following. 

Preferably the control system is adapted to control the linear axis drive to move in discrete 
steps each of a predetermined equal size, and in that event each linear axis movemem 
between exposures may be made up of one or more than one of the said discrete steps. 

Alternatively the Imear axis motion may be continuous. This allows faster motion between 
different unaging positions without having to wait for the mechanism to settle. The 
disadvantage is that the imagmg plane wUl vary during the imaging process with the result 
that the image can be blurred. However this may be compensated for by applying 
deconvolution to re-sharpen the unage. 

Where the time required to alter the linear axis spacing is greater than the minimum time 
needed between exposures the control means is adapted to control the time between 
exposures or to control the illumination and image capture device exposure by reference to 
the Z-axis movement such fliat excitation illumination and exposure is only initiated after a 
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sufficient period of time has elapsed, or after desired linear axis movement has been 
achieved. 

Multiple images may be collected over short periods of time (eg 1 minute-60 minutes) or 
extended periods of time (eg 24-72 hours) as well as intermediate periods (eg 1-24 hours). 
Such images are often referred to as a stack of images. 

Where the method involves variation of the wavelength of the excitation light from one 
exposure to another, the apparatus may comprise two or more excitation light sources each 
producing light of a different wavelength from the, or each, other source, or may comprise 
a single source which is adjustable to produce light of different wavelengths, and the 
control means is adapted to select between the source, or to control the single source, to 
alter the excitation light wavelength, preferably between the end of one exposure and the 
beguming of the next. 

Where the variation of illumination wavelength is to be effected whilst focussing die light 
onto the same plane of the specimen (ie there is to be no linear axis movement between 
each of a succession of two or more exposure steps making up a sequence of said steps, 
albeit each employing one (or simultaneously two or more) different wavelength(s) of 
excitation light the control means may be adapted to control the linear axis drive means so 
that linear axis movement is initiated only after each complete sequence of said steps has 
been performed, and the next sequence of said steps is only initiated after each linear axis 
movement has been completed. 

Typically the specimen is carried by a microscope stage and a stage drive is provided to 
move the stage (and specimen) in X and Y, to bring into view other portions of a sample 
such as for example other cells on the same slide, or other samples in a multiwell plate, or 
other regions of tissue or of a single sample. To this end the X or Y steps may be in the 
range 10 microns to 10 nrni. 
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A number of samples may be held in die wells of a multiwell plate. The samples may all 
be similar but may be different and may for example differ due to constituents, 
concentration or age. 

The linear drive may be controlled so as to produce linear motion alternatively in one 
direction, and then in another, to save time. 

Wavelength variations may be achieved by employing two or more light sources of 
different peak wavelengths and selectively operatmg the sources, or selectively directing 
light from the sources in turn, as required, to the scanning device and specimen. 

It is to be understood that the spectrum of a source of light (even when designated as 
producing light of one wavelength), actually includes light at a number of wavelengths, in 
which the intensity of the light at and close to the designated wavelength is much greater 
than other components of light in the source spectrum. In that event the source is said to 
peak at the designated wavelength. 

Thus the light sources used in apparatus for performing the invention may have more 
complex spectral characteristics, for example multiple peaks available from an Argon ion 
laser with a main line at 488nm, and additional lines at 476nm and 496nm« 

Where the excitation light source produces light of more than one wavelength at the same 
time. Individual (peak) wavelengths may be selected by a single code (eg 2 lines to select 
one of 4 wavelengths as code 00, 01, 10, 11) or as a set of individual signal lines (eg 4 
lines for 4 wavelengths), so that several lines may be activated at the same time. 

According to a preferred feature of the invention the excitation light source may comprise 
a laser light source, which may comprise one or more than one laser. 
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A single excitation light source may be employed, whose emitted wavelength can be 
altered, and the control means may be adapted in use to alter the source to produce a 
desired wavelength. 

Preferably a laser light source is employed which comprises an acousto-optical tuneable 
filter (AOTF) crystal, and the control means is adapted to provide in use signals to alter 
the frequency controlling signal to the crystal, between exposures, to control the 
wavelength of the emitted light. 

The light source may be a directly driven laser diode. 

Preferably any alteration of the frequency of the controlling signal is effected between 
exposures. 

The excitation light may be pulsed as by pulsing the source. 

The excitation light may be polarised, either by using a polarised light source or by using a 
polarising filter. 

The excitation light source may be a luminescent or incandescent source and an optical 
filter or a monochromator may be employed to control the wavelength of the light emitted 
by the source in use. 

Alternatively a light emitting diode (LED) source, which may comprise one or more light- 
emitting elements, may be used as the excitation light source. 

Where the intensity of the excitation illumination incident on the specimen is to be adjusted 
from one exposure to another, the control means may be adapted to adjust in use the 
intensity of the excitation illumination incident on the specimen, preferably between 
exposures. 
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This variation may for example be achieved by interposing different neutral density filters 
in the light path. 

Therefore apparatus may comprise a plurality of neutral density filters and means under 
the control of the control means, which in use selectively positions one or more of the 
filters in the light path, (preferably during the time intervals between exposures) to achieve 
die density variation. 

Alternatively the apparams may comprise an adjustable iris diaphragm in the light pafli and 
drive means operated by the control means to open or close the iris as required, preferably 
in the time intervals between exposures. 

Alternatively and preferably the excitation light intensity may be controlled by adjusting 
the power supplied to the excitation light source, and to this end tiie control means may be 
adapted to generate signals for controlling tiie power source for the excitation light source, 
or the transfer of power thereto, as appropriate. 

An attenuating element such as an AOTF or LCD shutter may be used to control the 
intensity of flie excitation light incident on Uie specimen, die AOTF or LCD shutter being 
controlled by signals from the control means. 

The excitation light intensity may be controlled by using two or more such intensity 
varying techniques in combination, typically in series. 

Where the intensity of the excitation illumination at one wavelength is different from fliat 
at another wavelengUi such as may be due to inherent intensity variation as between one 
source and anotiier or between different modes of operation of a controlled excitation light 
source such as an AOTF crystal laser, or of die emission of a crystal due to excitation, the 
control means may be adapted to alter in use the intensity of the illumination between one 
wavelength and another so as to provide a predetermined intensity of illumination at die 
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specimen for each wavelength, so as to substantially remove variation in intensity as 
between one wavelength and another, which can otherwise occur. 

Typically but not exclusively, the control means is adapted in use to control the power to 
the excitation light source from one exposure to another as wavelength varies, so as to 
ensure a substantially similar level of excitation Ulumination (intensity) at the specimen 
durmg each exposure, which level is indq>endent of wavelength. 

A similar technique may be employed if the emission from the sample differs according to 
wavelength of the luminescence emitted, and in order to render the emitted light of similar 
intensity (irrespective of wavelength), the power adjustment and/or attenuation relating to 
the excitation light source, from one exposure to another, may also, or instead, be adjusted 
to provide substantially constant intensity luminescence irrespective of the wavelength 
used. 

The image capmre device may comprise any convenient light sensitive sensor. 

A preferred image capttire device is a CCD camera. Preferably a CCD camera is selected 
which provides a picmre signal in digital format as opposed to analogue format. 

Where the image capture device is a CCD camera, it is preferably cooled to increase the 
S/N ratio of die camera ou^ut signals. 

Alternative image capmre devices which may be employed are a CMOS camera, a CID 
(charge injection device) camera, an intensified or gated camera, a photomultiplier tube 
array, a photodiode array or an image capmre device employing an addressable 
microcolometer array or a chemical film camera may be used. 

Since the light presented to the image capmre device is capable of forming an image, the 
light to be supplied to the image capture device may be directed instead (or as well) to an 
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eyepiece which will allow the image to be viewed by the human eye, or to projection 
apparatus which is adapted to project an image onto a screen for viewing. 

To this end the light path to the image capture device may include a beam-splitting or 
beam directing device by which light can be split (or divided) between the image capture 
device, and/or an eyepiece and/or a projection system. 

One or more image capture devices may be employed each of which may be the same type 
of device or may be different. Where more than two image capture devices are employed, 
at least one may differ in type from one or more of the others, and they may all be 
different. 

Cameras may be replaced by sensors. 

Where one or more image capture devices are provided, the light path thereto may include 
a beam splitting means or beam directing means as appropriate. 

The light may be transmitted to the or each image capture device sensor via an optical 
fibre bundle. 

The apparatus may include a data storage means adapted to store signals from the image 
capture device for subsequent analysis by a computer, or displayed on a screen for visual 
investigation or analysis of an image produced by reading out the stored signals. 

The apparams may also include computer means adapted to receive the signals from the 
image capture device or the data storage means, to permit processing and analysis of the 
image signals. 

A signal path to a display device from the image capture device, or the computer may be 
provided so that an image can be formed using the image capture device output or the data 
stored in the storage means or from the computer before and/or after processing. 
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It is to be understood that in place of a laser and AOTF, or a shutter, to control excitation 
light, a diode laser source may be employed which has beam control built in. In this 
context reference to a shutter can be considered to refer to switching or modulating a diode 
laser source. 

The invention will now be described by way of example with reference to the 
accompanying drawings in which: 

Figure 1 is a schematic diagram of a Confocal microscope system; 

Figure 2 is a system diagram giving details of proprietary hardware which can be 
connected to the computer based controller of Figure 1 to enable the method of the 
invention to be performed; 

Figure 3 is a schematic diagram of an AOTF laser control system; 

Figure 4 is a block schematic diagram showing how the system of Fig 2 is switched 
through a sequence of different machine states; 

Figure 5 is a block schematic diagram of the controller board; 

Figs 6A-6C are waveform diagrams showing control signal protocols for different 
operating systems; 

Fig 7 is a state diagram of a system operated as a phase locked loop; 

Figs 8A and 8B are system and state diagrams of a system in which further control signals 
are generated by the controller; 
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Figs 9A and (b are system and state diagrams of a system in which triggering of a camera 
generates trigger acknowledgement signals for feedback to the controller; and 

Figs lOA and lOB are system and state diagrams of a system in which the controller is 
externally triggered or can allow inputs to be used for external devices. 

Reference is made to of the Yokogawa Technical Report No. 33 (2002) and in particular 
Figures 1 to 3 of that Report and the related description, for a description of the operation 
of a CSUlO con-focal scanning system. 

In Figure 1 the con-focal scanning system 10 may comprise a Yokogawa confocal scanner 
(which may be a model CSU 10 or the more recent model CSU21), set to view a sample 
(not shown) on a stage 12 and provide light to a camera 14 to form an image thereon of 
fluorescence arismg from excitation of the sample by laser light from an AOTF laser 
source 16. 

A controller 18 converts signals from a computer 20 into control signals for operating or 
controlling the operation of items 10, 12, 14 and 16 and in turn can transmit data from 
these items to the computer 20, if required. Software 22 is shown loaded on the computer 
enabling the latter to be instructed to perform particular tasks or control items 10 to 16 in 
different ways, depending on the sample or experiment to be performed. Alternatively 
different software can be loaded as required to enable the system to perform different tasks 
or experiments. 

Figure 2 shows the various items of proprietary hardware which can be interconnected to 
constitute an operating example of the system of Figure 1. The controller 18 of Figure 1 is 
made up of the synchronisation circuit 18* and control board 18" in Figure 2, and the 
computer 20 is identified as a Dell Dimension GX260 loaded with Windows 2000 as the 
basic operating system, having a 2.4 GHz processor and 2GB of RAM and a WD Caviar 
drive, with a dual monitor card for driving two display monitors 21 and 23. 
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The microscope 12 is typically a Nikon eclipse model TE300 and as mentioned in relation 
to Fig 1 the con-focal scanning unit comprises a Yokogawa model CSUlO or CSU21. 
Z-axis adjustment is achieved using a Physik Instrument piezoelectric driver 22 which is 
controlled by a proprietary control unit 24 provided with analogue control signals fromlS. 

The laser excitation light source is an Omnichrome Series 43 (3 wavelengtii) laser as 
supplied by Melles Griot Laser Group of California operating with an AOTF crystal and 
controller from NEOS of Florida. The light intensity of the laser is controllable from the 
Control board 18" which (as indicated above) together witii the synchronisation unit 18' 
makes up the controller 18 of Figure 1. A driver 26 controls the AOTF control 28 for the 
laser 16 whose power supply is shown at 30. 

The camera 14 is a Hamamatsu Orca EP CCD camera. 

A power supply for the driver 26 is shown at 32 and a power supply and interface for the 
camera 14 is shown at 34. 

The synchronisation circuit 18' is connected to the computer 20 via a USB connection 36. 

Figure 3 shows details of die AOTF laser, as comprising an Argon ion laser 38 and 
auxiliary laser 40. An AOTF crystal 42 conbrols the wavelength of the light emitted via 
fibre 44 and ttie CSUlO fibre port. 

The crystal 42 is controlled by an RF signal from the controller 46 which in turn is 
conti-oUed by TTL and analogue signals from the computer 20 via circuit 18* and driver 26 
(see Figure 2). 

The State diagram of Figure 4 shows how the 1 KHz clock signal controls the different 
operations required of the system. 
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Figure 4 is a state diagram containing a set of ouq)ut registers labelled line on/off (48), 
laser line (50), Z position (52), camera (54), and head sync (56); a duration down counter 
(58), a state counter (60), and a memory 62 containing a plurality of separate bit pattern, 
each pattern defining one of the desired machine states. The bit outputs of some of the 
output registers are connected directly to digital ports on devices which can be controlled 
by such signals, such as the laser line ON/OFF selection 48 systems, the camera trigger 
aiKl the CSU scannuig disc system synch input. Digital signals from ou^ut registers 50 
and 52 are converted from digital to analogue signals by DAC's 64, 66 to provide 
analogue signals to analogue ports on the AOTF system and the piezo stage drive, which 
typically drives the stage along two difrerent axes. 

A clock signal, typically of IKHz, is supplied to the duration downcounter 58. 

Typically the state data is stored in the host computer 20 and is loaded into the memory 62 
from the host computer. 

At initialisation, the state counter 60 is loaded with the number of states. The first state is 
transferred from the first line in memory 62 to the output registers 48 to 56, so as to set 
the states of the devices supplied with control signals therefrom. The intended duration of 
the first state is loaded as a number into the state duration down-counter 58 from state 
duration memory 68. 

At the next clock pulse, the numerical value held in the duration counter 58 is decremented 
by one, and its new value compared with zero. If non-»ro, the states of the ou^ut 
registers 48-56 are maintained. 

When the numerical value in the duration counter 58 reaches zero, the state counter 60 is 
clocked by one and now points to flie next state bit pattern in the memory 62. This next 
state is transferred from 62 to the output registers 48 to 56. so as to set the states of the 
devices they control to the new condition required of the machine and the duration for this 
next state is transferred from 68 to the duration counter 58 to confrol the duration of fliis 
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next state, which is replaced by the next state from memory 62 each time the duration 
counter 58 numerical value reaches 0. 

The system operation progresses from one state to the next imtil the state counter 60 is 
decremented to zero at which point the operation stops. A stop signal may be generated at 
this pint and the system may be set up to start again upon generation of this stop signal. 

Messages relevant to different operations/states are set out below. 

U$B Message Typgs 
Imgrrpgation 

This message is a work around to cope with initial lack of a USB vendor ID. The USB 
master interrogates each unit to find out what ^e it is. 



Message ^e 


0x00 


1 byte 


Filler (usually all 0x00) 


3 bytes 




4 bytes 


The synchroniser replies with: 


Message type 


QxOO 


1 byte 


Second byte of interrogation message echoed (i.e. 0x00) 


1 byte 


0x12 (i.e. ASCII DC2) 


1 byte 


0x36 (i.e. ASCII '6') 


1 byte 




4 bytes 



InitiaUsatiPn 

This message is sent before any experiments are conducted, to set the default output state 
and the confocal head parameters. 



Message type ' 


1 byte 


Digital outputs (Camera trigger bit 7, bits 0 to 6 general purpose) 


1 byte 


AOTF bits 


1 byte 


Z position (see note 1) 


2 bytes 


Sync pulse on period, ms (see note 2) 


1 byte 


Sync pulse off period, ms (see note 2) 


1 byte 
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Sector time, ms (see note 2) 


1 byte 




8 bytes 



7-axis drive 

Z-axis drive position is given by: 

p2/(100/im/2'«) 
Or 

Pz/(1.526mn) 



Where Pz is die desired Z position and a sensitivity of lO/xm/volt and a 10 volt ouQ)ut span 
is assumed. Up to 16 bits are available for this value, but tiie embodiment being described 
responds only to the most significant 12 bits, which gives a resolution of 2Snm. 

Sector time sychronisarion 

Sync pulse times, number of sectors and camera exposure times are all constrained by the 
requirement for every event to be synchronised to sector times. 



For example, a 12 sector disk could have tiie following parameters. 

DiskRPM 5000 2500 1666.66 

7 

Disk Hz 83.3333 41.6666 27.7777 
3 7 8 

Sector time ms:. ' '• 1.. ; .2 . 3 

Sync Hz 166.666 83.3333 55.5555 
7 3 6 

Sync period ms 6 12 18 

The Yokogawa CSUlO and CSU21 systems would be limited to 3ms per sector in the 
present design. 



State data header and footer 



The following message is sent before (header) and after (footer) the list of State Data. Any 
State Data that is already stored is cleared when the header is received. The message is 
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ignored if the synchroniser is running. The messages are referred to as State list headers 
and State list footers. 



Message type 


H 


1 byte 


Sub-type 




1 byte 


• H: Header 






• F: Footer 








2 bytes 



Sta^.Qata 



State data should immediately follow a state list header. The synchroniser is set to check 
that the correct number of states are received before executing anything. 



Message type ^ 


1 byte 


Time for this state to persist (in sector widths) 


2 bytes 


Digital outputs (Camera trigger bit 7, bits 0 to 6 general purpose) 


1 byte 


AOTF bits 


1 byte 


Z position 


2 bytes 




7 bytes 



Experiment CQntroll 

Once State Data has been downloaded, the system can be controlled to perform an 
experiment. 



Message type ^ 


1 byte 


Action 
code: 

L Start live mode (immediate data is accepted and retained) 

0 Start experiment to run once (immediate data is ignored) 
S Start experiment in continuous loop (immediate data is 

ignored) 

C Stop cleanly (stop execution at the end of the state list) 

1 Stop immediately (stop everything and revert to the 
initialisation data) 


Ibyte 




2 bytes 
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Tmmediate data 

If capture is started in live mode, the digital outputs, AOTF blanking and Z stepper may 
be altered at will. The camera trigger bit (digital ou^ut bit 7) is ignored in all immediate 
data. 



Message type 




1 M 


1 byte 


Data type: 
C 

A 

Z 


Digital ou^ut data (1 byte follows) 
AOTF blanking data (1 byte follows) 
Z stepper position (2 bytes follow) 




1 byte 


Data 


2 bytes 




5 bytes 



AOTF/DAC control 

This message is sent to set the AOTF channel power. It would not be sensible for the 
AOTF levels to be changed during an experiment, but for simplicity the synchroniser wUl 
always respond to them - this enables the levels to be tweaked while in live mode. 



Message type 


1 A 


I byte 


AOTF address (1 to 8) 


1 byte 


AOTF data 


1 byte 




3 bytes 



state 


line on/off 


laser line 


Piezo 


earner 
a 


sync 


count 


0008 


0000 


0000 


0110 


0 


1 


0001 


0007 


0010 


0200 


0110 


0 


0 


0001 


0006 


0010 


0200 


0110 


1 


0 


0002 


0005 


0010 


0200 


0110 


0 


0 


0001 


0004 


0010 


0000 


0110 


0 


0 


0001 


0003 


0001 


0100 


0330 


0 


0 


0001 


0002 


0001 


0100 


0330 


1 


0 


0001 


0001 


0001 


0100 


0330 


0 


0 


0001 


0000 


0001 


0100 


0330 


0 


0 


0001 
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State machi ^f^ ^amplft content 

The system carries out a given state for 'count* clock ticks and outputs persist at given 
value for this time. 

As described the invention provides a control system or synchroniser to sequence a 
Nipkow disc laser scanning confocal microscope imaging system having a Z-axis drive. 
The system requires a host computer (20) and controller (18) and is programmable to 
perform a wide range of tasks and is connectable to a Nipkow disc confocal microscope 
hardware via a communication line such as a USB connection. 

The controller (18) may be constructed as shown in Figure 5. using a 8051 microcontroller 
which is programmed to function as the state machine described with reference to Figure 
4. The unit (18) includes a clock input 69, the DAC's 64 and 66 and output ports as 
follows, (although it is to be understood diat any other suitable microprocessors or 
programmable logic device could be used to control the subsystem), namely 

• USB port 70 - one port for configuration, tuning and power. 

. Sync output port 72 - variable 0-3Vpk-pk, programmable high and low periods in 1ms 
increments, as described with reference to Initialisation 

• TTL output ports 74-8 bits for AOTF switching; 67 employing up to 8 bits for 
camera trigger, shutter control and/or general purpose use; (For example one bit may 
be dedicated to camera triggering and the rest may be controUed asynchronously when 
in live mode); and 72 for synchronising disc sector speed. 

. An analogue output from DAC 64 - 8 chamiels, 0-5 volts. 8 bit resolution. For AOTF 

power setting and/or general use. Not synchronised, but may be controlled 

asynchronously or synchronously, and 
. An analogue output from DAC 66 - 0-10 volts. 12 bit resolution. Typically 

synchronised to disc sector speed, but may be controlled asynchronously when in live 

mode. 
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In the control signal protocol diagrams of Figs 6A, 6B and 6C the values of the control 
signals are shown for a typical system such as described with reference to Figs 2-5. 

The trigger/enable signal controls the activation of the camera, which alternates between 
active "on" and inactive "ofT states. 

The "end of scan sync" indicates the end of the pinhole scan sequence corresponding to an 
entire disc rotation, where each disc consists of a number of sectors. 

"Wavelength select" indicates the different wavelengths selected for excitation. 

"Power lever indicates when the power is ON and OFF. This is typically switched from 
OFF to nominal power level "p". 

The "Z position" waveform of Fig 6A indicates how the control signal determining the 
different sample scan planes in the Z axis can increase in steps from Zl to Z2 and Z3 
before returning to a ZO position ready for the next scan sequence, while the wavelengdi 
remains constant. 

In Fig 6B the Z position determining signal ramps at a constant rate from ZO to a 
maximum value for Z (i.e. X3) and then rapidly reduces in value back to XO ready for tiie 
next scan. 

Fig 6C shows how the Z position signal varies between Zl and Z3 during two successive 
scans during each of which the wavelength is altered from XI to X2 during a power OFF 
interval 

Fig 7 shows tiie states of a system such as tiiat of Figs 2-5 when operated as a phase 
locked loop about the natural frequency of the disc rotation pulse, where the disc rotation 
speed is set by die CSU controller. In this system, the rate of the clock to die counter is 
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adjusted by the actual rate of disc rotation. Disc sync signals are supplied to a phase locked 
loop clock 76 which controls the duration counter 58 in place of the 1 KHz clock of Fig 4. 

Figs 8A and SB relate to an alternative arrangement in which iurther control signals pass 
frooi the controller 18 and the microscope 12 for other control fimctions such as stage and 
objective control as depicted by the confirm signal path 78. 

Figs 9A and 9B relate to another modification of the system of Figs 2-5, in which further 
control signals pass from the camera 14 to the controller 18 to acknowledge triggering, 
such as an acknowledge signal along path 80. 

Figs lOA and lOB show how the controller 18 can be externally triggered or can receive 
an enable input from an outside device along path 82. 



